Papers with defense techniques
In and Out-of-Domain Text Adversarial Robustness via Label Smoothing (2023.acl-short)
Copied to clipboard
| Challenge: | Existing studies show that state-of-the-art NLP models are vulnerable to adversarial attacks . label smoothing has been proven effective in a variety of applications and modalities . |
| Approach: | They propose to use label smoothing to improve adversarial robustness in pre-trained models against various popular attacks. |
| Outcome: | The proposed method significantly improves adversarial robustness in pre-trained models against various popular attacks. |
Attacking Vision-Language Computer Agents via Pop-ups (2025.acl-long)
Copied to clipboard
| Challenge: | Existing tools for analyzing and testing VLMs are lacking in understanding what types of attacks are possible and what types are effective. |
| Approach: | They propose to integrate pop-ups into existing agent testing environments to attack VLM agents by ignoring them. |
| Outcome: | The proposed attack success rate is 86% and decreases by 47% when integrating pop-ups into existing agent testing environments. |
A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated capabilities for generating content that could be deemed harmful. |
| Approach: | They conduct a comprehensive analysis of existing studies on jailbreaking LLMs and their defense techniques. |
| Outcome: | The proposed techniques underperform existing white-box attacks and include special tokens significantly affects the likelihood of successful attacks. |
Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) are susceptible to jailbreak attacks, authors say . multimodal information increases the risk of attacks, but also provides additional data . |
| Approach: | They propose a jailbreaking detector that detects maliciously perturbed image inputs . cross-modality information detector is designed to detect cross-modal similarity between harmful queries and adversarial images. |
| Outcome: | a new tool can detect maliciously perturbed image inputs without modification or computation cost. |
Defense Against Prompt Injection Attack by Leveraging Attack Techniques (2025.acl-long)
Copied to clipboard
| Challenge: | Recent attacks leverage LLMs’ instruction-following abilities and their inabilities to distinguish instructions injected in the data content. |
| Approach: | They invert the intention of prompt injection methods to develop novel defense methods based on previous training-free attack methods by repeating the attack process with the original input instruction rather than the injected instruction. |
| Outcome: | The proposed methods outperform existing defense approaches, achieving state-of-the-art results. |